7  Dashboards

Author

Gabriele Filomena

Published

March 20, 2024

8 Building Dasbhboards

The Lecture slides can be found here.

This lab’s notebook can be downloaded from here.

## Run only if necessary
# !pip install panel
# !pip install hvpolt
# Standard library imports
import datetime as dt

# Third-party imports
import geopandas as gpd
import hvplot.pandas
import numpy as np
import pandas as pd
import panel as pn
import plotly.express as px

# Initialize Panel with extensions
pn.extension('plotly', design='material')

8.1 Importing the Data

df = pd.read_csv('../data/GTD_2022.csv', low_memory=False)
## visualise main columns
df[['gname', 'year', 'date', 'country_txt', 'nkill', 'nwound','weaptype1_txt']].head()
gname year date country_txt nkill nwound weaptype1_txt
0 Unknown 2005 28/05/2005 Pakistan 1.0 0.0 Firearms
1 Ansar al-Sunna 2005 29/05/2005 Iraq 1.0 0.0 Firearms
2 Al-Qaida in Iraq 2007 08/06/2007 Iraq 15.0 0.0 Firearms
3 Taliban 2010 15/06/2010 Afghanistan 1.0 0.0 Firearms
4 Communist Party of India - Maoist (CPI-Maoist) 2011 06/01/2011 India 1.0 0.0 Firearms

8.2 Creating a Basic Dashboard: Map + Date Slider

We are going to use Panel a Python library for creating interactive and dynamic web-based dashboards and applications. It allows data scientists, analysts, and so forth, to turn data sets into interactive dashboards using a wide array of widgets, plots, and layouts without requiring deep web development skills. Panel supports various plotting libraries like Matplotlib, Bokeh, and Plotly, as well as our friend Folium, and allows working with Pandas, NumPy, and others. It’s flexible enough to serve either as a standalone app or embedded in existing web applications, and it can be deployed easily. Have a look at this article for some insights on the power of Panel

df['date']= pd.to_datetime(df['date'], dayfirst=True)
date_slider = pn.widgets.DateSlider(
    name='Date',
    start=df['date'].min(),
    end=df['date'].max(),
    value=df['date'].min()  # Single date value
)

df['date']= pd.to_datetime(df['date'], dayfirst=True).dt.date # this is not very handy but otherwise the slider would not communicate with the df

And here’s the slider

date_slider

Now, we create a function to update the map, on the basis of the slider.

def update_map_date(selected_date):
    # Filter the DataFrame based on the selected date
    filtered_df = df[df['date'] == selected_date].copy()
    # Create the scatter geo plot
    fig = px.scatter_geo(filtered_df, lat='latitude', lon='longitude')
    return fig
# Bind the update_map function to the date_slider, vertical layout
interactive_panel = pn.Column(date_slider, pn.bind(update_map_date, date_slider))

# Serve the Panel dashboard
interactive_panel.servable()

dashboard.servable() makes the dashboard “servable,” meaning it can be run as a standalone app or served via a notebook, depending on how you’re using Panel. When you call .servable() on a Panel object, you’re essentially telling Panel that this is the object you want to display when the dashboard is run. If you’re working in a Jupyter notebook, this allows the dashboard to be rendered inline.

8.2.1 The bind function

In Panel, pn.bind is a function that creates a dynamic link between widget values and a function, enabling interactive and reactive applications without the need for explicit callbacks or event handlers. When you use pn.bind, you’re essentially telling Panel to watch certain parameters (like the value of a widget) and call a specified function whenever those parameters change, automatically passing the new values to the function.

Here’s a breakdown of how pn.bind works: - pn.bind links a function to one or more parameters (often widget values). This function will be called whenever the linked parameters change. - When the linked parameters change, pn.bind automatically calls the specified function with the new values as arguments. This eliminates the need for manual event handling or value extraction within the function. - This binding creates a reactive link, meaning the output of the function will automatically update in the UI when the input parameters change. This is fundamental for creating interactive and dynamic dashboards.

Usage with Layouts: When you use pn.bind within a Panel layout (like pn.Column or pn.Row), the return value of the bound function is dynamically inserted into the layout. If the function returns a plot, a table, or any other visual component, that component will update in the UI whenever the function is triggered by a change in the bound parameters. Here’s a simple example to illustrate:

# Define a slider
slider = pn.widgets.IntSlider(name='Slider', start=0, end=10, value=5)

# Define a function that takes a parameter and returns a value based on that parameter
def multiply_by_two(x):
    return x * 2

# Bind the function to the slider's value
bound_function = pn.bind(multiply_by_two, slider.param.value)

# Create a layout that displays the slider and the output of the bound function
layout = pn.Column(slider, bound_function)
layout.servable()

pn.Column(...) creates a vertical layout (column) containing the row with certain elements/widgets/panes. pn.Column is used when you want to stack components vertically. Contrarily, pn.Row(): creates a horizontal layout (row). pn.Row is used when you want to place components side by side horizontally. In both cases you can add one or more elements.

8.3 Some More Widgets

Panel offers a range of widgets for interactive user inputs. These widgets can be used to receive input from users and update the dashboards/panels accordingly. Have a look here for a list of other widgets and usage examples.

# Select widget for choosing a city
group_selector = pn.widgets.Select(name='Group', options=list(df.gname.unique()))

# RangeSlider for selecting a numeric range
year_slider = pn.widgets.IntSlider(name='Year', start=df.year.min(), end=df.year.max(), step=1)

# RangeSlider for selecting a numeric range
range_slider = pn.widgets.RangeSlider(name='Nr of People Killed', start=df.nkill.min(), end=df.nkill.max(), step=1)

# CheckBox for a boolean choice
check_box = pn.widgets.Checkbox(name='Check Me')

# TextInput for freeform input
text_input = pn.widgets.TextInput(name='Enter Text')

# RadioButtons for exclusive selection
radio_button = pn.widgets.RadioButtonGroup(name='Options', options=['Option 1', 'Option 2', 'Option 3'])
# Layout these widgets in a column
widgets_column = pn.Column(group_selector, year_slider, check_box, text_input, radio_button)
widgets_column

8.3.1 Subsetting the Dataframe on the basis of categorical variables

# Create a dataset just for Iraq (feel free to change it)
iraq_df = df[df.country_txt == 'Iraq'].copy()
# Calculate the centroid of the filtered points for centering the map
center_lat = iraq_df['latitude'].mean()
center_lon = iraq_df['longitude'].mean()
# Create a Select widget for the 'group' column
group_selector = pn.widgets.Select(name='Group', options=iraq_df['gname'].unique().tolist())

# Define a function to update the map based on the selected group
@pn.depends(group_selector.param.value)
def update_map_iraq(selected_group):
    # Filter the DataFrame based on the selected group
    filtered_df = iraq_df[iraq_df['gname'] == selected_group].copy()
    
    # Generate the map
    fig = px.scatter_geo(filtered_df, lat='latitude', lon='longitude', title=f"Attacks carried out by: {selected_group}",
                        center={"lat": center_lat, "lon": center_lon})

    # Adjusting the map's view to a 'closer' zoom
    fig.update_geos(projection_type="natural earth", lataxis_range=[center_lat-10, center_lat+10], lonaxis_range=[center_lon-20, center_lon+20])
    # Return the figure
    return fig

# Create a Panel layout to display the widget and the map
dashboard = pn.Column(group_selector, update_map_iraq)

# Display the dashboard
dashboard.servable()

Plotly Express is here used to generate a geographical scatter plot.

px.scatter_geo: This creates a scatter plot on a geographic map. The arguments lat='latitude' and lon='longitude' specify the DataFrame columns that contain the latitude and longitude coordinates for the points to be plotted. The title argument sets the title of the map, and center specifies the central point of the map view, ensuring the map is centered around the points of interest. fig.update_geos updates the geographic layout of the figure. It’s used here to adjust the map’s projection and zoom level.

  • projection_type="natural earth": Sets the map’s projection type to “natural earth,” which is a visually appealing and commonly used projection for world maps.
  • lataxis_range=[center_lat-10, center_lat+10]: Defines the range of latitude to be displayed on the map. This setting zooms in on the region by limiting the latitude range to 10 degrees above and below the center latitude.
  • lonaxis_range=[center_lon-20, center_lon+20]: Defines the range of longitude to be displayed on the map. Similar to lataxis_range, this limits the longitude range to 20 degrees on either side of the center longitude, effectively zooming in on the area of interest.

We will be switching to folium in a bit, also for the sake of continuity and because it’s more powerful, so don’t worry to much about it.

columns = ['gname', 'year', 'date', 'country_txt', 'nkill', 'nwound','weaptype1_txt']

df = df.dropna(subset=['latitude', 'longitude'])
# Widget to select a country
countries = sorted(df['country_txt'].unique().tolist())
country_selector = pn.widgets.Select(name='Country', options=countries)

8.4 Complex Layouts

Layouts in Panel are used to organize widgets and plots in a structured manner. We are mainly working with pn.Row() and pn.Column() but familiarise yourself with other possible layouts (see here).

# still using the country selector here

new_country_selector = pn.widgets.Select(name='Country', options=countries)

We can add a scatter plot to the dashboard

# Plot
def update_scatter(data, width, height, title):
        return data.hvplot.scatter(
        x='year', 
        y=['nkill', 'nwound'],  # Ensure these column names match your DataFrame
        title=title,
        width=width,
        height=height
    )

And a function that updates several panels based on the country selector. See what happens when you select another country from the list.

@pn.depends(new_country_selector.param.value)
def update_dashboard(country):
    data = df[df['country_txt'] == country].copy()
    
    # Creating a plot for number of attacks over time
    plot = update_scatter(data, 800,300, 
                          title = f'Number of People Killed and Wounded Over Time in {country}').opts(legend_position='top_left')
    
    # Creating a summary table
    table = pn.widgets.DataFrame(data[columns], show_index=False, width=600)
    
    return pn.Column(plot, table)

# Layout the dashboard
dashboard = pn.Column(
    pn.Row(country_selector),
    update_dashboard
)

dashboard.servable()

8.4.1 Using Folium Maps

As mentioned, Panel allows us to incoporate folium maps.

# let's reset variables to avoid interactions between different functions/dashboards
%reset -f -s

# remporting again
import geopandas as gpd
import hvplot.pandas
import numpy as np
import pandas as pd
import panel as pn
import plotly.express as px

# Initialize Panel with extensions
pn.extension('plotly', design='material')
df = pd.read_csv('../data/GTD_2022.csv', low_memory=False)
df = df.dropna(subset=['latitude', 'longitude'])
countries = sorted(df['country_txt'].unique().tolist())
country_selector = pn.widgets.Select(name='Country', options=countries)
# Function to create a map centered on the selected country
import folium
from folium.plugins import MarkerCluster

def create_foliumMap(data):

    # Calculate the mean latitude and longitude to center the map
    center_lat = data['latitude'].mean()
    center_lon = data['longitude'].mean()

    # Create a Folium map centered on the average location
    folium_map = folium.Map(location=[center_lat, center_lon], zoom_start=6)

    # Use a MarkerCluster to add markers for each event
    marker_cluster = MarkerCluster().add_to(folium_map)

    # Add a marker for each event
    for idx, row in data.iterrows():
        folium.Marker(
            location=[row['latitude'], row['longitude']],
            popup=f"Date: {row['date']}<br>Deaths: {row['nkill']}",
        ).add_to(marker_cluster)

    # Return the Folium map object
    return folium_map

This function communicates with the one above for updating the attributes used to create the Folium map.

# Panel doesn't directly render Folium maps, so we need to render it as HTML
def update_map_country(df, country, width, height):
    # Filter the DataFrame for the selected country
    data = df[df['country_txt'] == country].copy()
    folium_map = create_foliumMap(data)
    # Panel doesn't directly render Folium maps, so we need to render it as HTML
    return pn.pane.HTML(folium_map._repr_html_(), width=width, height=height)

Then we bind the function to the widget and pass the DataFrame, along with the size attributes.

width = 700
height = 500
map_pane = pn.bind(update_map_country, df,  country_selector.param.value, width, height)

# Layout the dashboard
dashboard = pn.Column(
    pn.Row(country_selector),
    map_pane
)

dashboard.servable()

You may have noticed the use of the @pn.depends decorator. In Panel this is used to create reactive functions, namely functions that automatically update their output when an input parameter changes. This is a core concept in creating interactive and dynamic dashboards with Panel.

8.5 Grid Layout

GridSpec in Panel is a layout object that allows you to arrange your visual components in a grid-like structure, specifying the position and size of each component by defining rows and columns. You can access and assign components to specific areas of the grid using slicing syntax, e.g., grid[0, 0] = component places a component in the first row and first column of the grid.

Now, let’s try to combine the map, the scatterplot, and the dataframe panel in a unique dashboard. We are going to create a GridSpecLayout, see https://panel.holoviz.org/reference/layouts/GridSpec.html.

# let's reset variables to avoid interactions between different functions/dashboards
%reset -f -s

# remporting again
import geopandas as gpd
import hvplot.pandas
import numpy as np
import pandas as pd
import panel as pn
import plotly.express as px

# Initialize Panel with extensions
pn.extension('plotly', design='material')
df = pd.read_csv('../data/GTD_2022.csv', low_memory=False)
df = df.dropna(subset=['latitude', 'longitude'])
countries = sorted(df['country_txt'].unique().tolist())
country_selector = pn.widgets.Select(name='Country', options=countries)
# Function to create a map centered on the selected country
import folium
from folium.plugins import MarkerCluster

def create_foliumMap(data):

    # Calculate the mean latitude and longitude to center the map
    center_lat = data['latitude'].mean()
    center_lon = data['longitude'].mean()

    # Create a Folium map centered on the average location
    folium_map = folium.Map(location=[center_lat, center_lon], zoom_start=6)

    # Use a MarkerCluster to add markers for each event
    marker_cluster = MarkerCluster().add_to(folium_map)

    # Add a marker for each event
    for idx, row in data.iterrows():
        folium.Marker(
            location=[row['latitude'], row['longitude']],
            popup=f"Date: {row['date']}<br>Deaths: {row['nkill']}",
        ).add_to(marker_cluster)

    # Return the Folium map object
    return folium_map
    
# Panel doesn't directly render Folium maps, so we need to render it as HTML
def update_map_country(df, country):
    # Filter the DataFrame for the selected country
    data = df[df['country_txt'] == country].copy()
    folium_map = create_foliumMap(data)
    # Panel doesn't directly render Folium maps, so we need to render it as HTML
    return pn.pane.HTML(folium_map._repr_html_())
    
# Plot
def update_scatter(data, title):
        return data.hvplot.scatter(
        x='year', 
        y=['nkill', 'nwound'],  # Ensure these column names match your DataFrame
        title=title,
    )
countries = sorted(df['country_txt'].unique().tolist())
country_selector = pn.widgets.Select(name='Country', options=countries)

@pn.depends(country_selector.param.value)
def update_components(country):
    # Update the map
    map_pane = update_map_country(df, country)
    
    # Filter the data for the DataFrame pane
    data = df[df['country_txt'] == country]
    
    # DataFrame pane
    df_pane = pn.widgets.DataFrame(data)
    
    # Scatter plot
    plot = update_scatter(data, title=f'Number of People Killed and Wounded Over Time in {country}').opts(legend_position='top_left')
    
    # Use GridSpec for layouts
    grid = pn.GridSpec(sizing_mode='stretch_both')  # Changed to stretch_width
    grid[0, 0] = pn.Row(country_selector)
    grid[1, :2] = map_pane
    grid[2, 1] = plot
    grid[3, 1] = df_pane
    
    return grid

# Layout the dashboard
dashboard = pn.Column(update_components)
dashboard.servable()

Think about including other widgets to improve the visualisation. For some countries it’s impossible to look at the scatter plot at first, because the amount of records associated with it.

8.6 Alternative Widgets

Let’s play with another widget that we briefly mentioned above, the RadioButtonGroup widget.

# Create a dataset just for Iraq (feel free to change it)
iraq_df = df[df.country_txt == 'Iraq'].copy()
# Calculate the centroid of the filtered points for centering the map
center_lat = iraq_df['latitude'].mean()
center_lon = iraq_df['longitude'].mean()

# Function to create a Folium map based on the option selected (nkills or nwound)
def create_foliumMap(option):
    m = folium.Map(location=[center_lat, center_lon], zoom_start=6)  # Centered on Iraq

    for idx, row in iraq_df.iterrows():
        if option == 'Killed':
            folium.Circle(
                location=[row['latitude'], row['longitude']],
                radius=row['nkill'] * 100,  # Adjust size factor as needed
                color='red',
                fill=True,
                fill_color='red'
            ).add_to(m)
        elif option == 'Wound':
            folium.Circle(
                location=[row['latitude'], row['longitude']],
                radius=row['nwound'] * 100,  # Adjust size factor as needed
                color='blue',
                fill=True,
                fill_color='blue'
            ).add_to(m)

    return m

# Panel widgets for options
option_selector = pn.widgets.RadioButtonGroup(
    name='Select Option', 
    options=['Killed', 'Wound'], 
    button_type='success'
)
# Function to update the map based on the selected option
@pn.depends(option_selector.param.value)
def update_map(option):
    folium_map = create_foliumMap(option)
    return pn.pane.HTML(folium_map._repr_html_(), width=700, height=500)

# Dashboard layout
dashboard = pn.Column(
    option_selector,
    update_map
)

dashboard.servable()

This map is a bit slow in terms of reaction. Think about possible improvements or how you could tweak it (e.g. recreating the map everytime the button is pressed, etc.).

Also, consider using panel in combination with Bokeh or Geoviews. When integrating Geoviews with Panel for geospatial data visualizations, you can exploit its mapping capabilities alongside GeoPandas to create dynamic and interactive dashboards. Panel acts as a high-level framework that can host Geoviews plots, enabling the assembly of various visualization components, including maps, into a cohesive dashboard. You can use Geoviews plotting tools to render maps with different geometries and overlay them on tile sources. See for example here, or here

GeoViews is built on top of Bokeh and HoloViews. It simplifies the process of creating maps, allowing you to work directly with GeoDataFrames and use a variety of geometries (points, lines, polygons) for your visualizations. Check: https://geoviews.orgs.

Exercise:

Pick the dataset you used for Assignment I and spend around 30 minutes trying to create a dashboard that is effective at communicating some aspects your consider relevant of the dataset. As discussed in the lecture, start by thinking exactly what you want to communicate, and build from there.

Think about how to improve the dashboards above, include a widget whose use was not demonstrated above.

Presentation

You will then have 30 seconds to present your dashboard and hit the following point - What the dashboard shows - What interactivity/analysis element(s) you have used - One thing you think is really effective about it

Remember, 30 seconds. Short and sweet. Make them count